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PROJECT ABSTRACT 




Title: DEVELOPMENT AND EVALUATION OF AN EXPERIMENTAL CURRICULUM FOR THE 

NEW QUINCY (MASS.) VOCATIONAL-TECHNICAL SCHOOL 

Beginning and Ending Dates: 1 September 1968-31 August 1970 



OBJECTIVES: The principal goal of the project is to demonstrate increased 

effectiveness of instruction whose content is explicitly derived from analy- 
sis of desired behavior after graduation, and which, in addition, attempts 
to apply newly developed educational technology to the design, conduct, and 
evaluation of vocational education. Included in this new technology are 
methods of defining educational objectives, deriving topical content for 
courses, preparation of students in prerequisite knowledges and attitudes, 
individualizing instruction, measuring student achievement, and establishing 
a system for evaluating program results in terms of outcomes following gradu- 
ation. 

PROCEDURE: The procedure begins with the collection of vocational infor- 

mation for representative jobs in 11 different vocational areas. Analysis 
will then be made of the performances required for job execution, resulting 
in descriptions of essential classes of performance which need to be learned. 
On the basis of this information, a panel of educational and vocational 
scholars will develop recommended objectives for a vocational curriculum 
which incorporates the goals of (a) vocational competence; (b) responsible 
citizenship; and (c) individual self-fulfillment. A curriculum then will be 
designed in topic form to provide for comprehensiveness, and also for flex- 
ibility of coverage, for each of the vocational areas. Guidance programs 
and prerequisite instruction to prepare junior high students will also be 
designed. Selection of instructional materials, methods, and aids, and 
design of materials, when required, will also be undertaken. An important 
step will be the development of performance measures tied to the objectives 
of instruction. Methods of instruction will be devised to make possible 
individualized student progression and selection of alternative programs, 
and teacher-training materials will be developed to accomplish inservice 
teacher education of Quincy School Personnel. A plan will be developed for 
conducting program evaluation not only in terms of end-of-year examinations, 
but also in terms of continuing follow-up outcomes after graduation. 
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Project ABLE 

THE DEVELOPMENT AND EVALUATION OF AN EXPERIMENTAL CURRICULUM 
FOR THE NEW QUINCY (MASS.) VOCATIONAL-TECHNICAL SCHOOL 



Introduction : If today one were asked to characterize, in one word ed- 

ucation in the United States, that word would likely be "change." Never 
has education been of such national concern, with tremendous pressures at 
all levels to improve the product of our educational processes. Yet while 
we read on the one hand of the various Federal acts which provide assistance 
for education, there appears to be a continuing problem with our youth 
leaving high school prior to graduation. Perhaps of even greater concern, 
large numbers of our high school graduates enter the labor market with little 
or no employable skill for today's industrial technology. By some estimates 
as much as 75% of our secondary school student population will not go on to 
college; an alarming portion of these non-college bound youngsters will 
follow a general curriculum in high school which fits them for no specific 
employment without further formal training. 

Paradoxically, industry at this time has large numbers of unfilled jobs 
at the middle skill levels, and automation, while eliminating some jobs, is 
also creating jobs at low and upper middle levels. It does not seem too 
much to ask of education to prepare more of our youth for gainful application 
of their high school studies. As early as 1963, administration in the school 
system of Quincy, Massachusetts, took a positive step toward changing the 
output of its secondary schools. Based on a report from the Massachusetts 
Department of Education, Division of Vocational Education, that a new voca- 
tional-technical school should be established, the School Committee approved 
plans for the new facility. Planning for a curriculum which departed in a 
number of aspects from the traditional ideas contained in the usual "trade" 
school was initiated. The program and architecture of the new vocational - 
technical school in Quincy was built around the concepts of job families. 



Purpose : The development of a program of study for the new school is a 

joint research project of the Quincy Public Schools and the American Insti- 
tutes for Research, Pittsburgh, Pennsylvania. It is sponsored by the U. S. 
Office of Education and is scheduled over a five-year period from its begin- 
ning in April 1965« The staff includes twelve full-time research people 
from the American Institutes for Research and twenty faculty members from 
the Quincy Public Schools who combine half-time on the project and half-time 
in teaching. In addition, teachers throughout the system provide a special 
assistance from time to time. A panel of seven advisors, each of whom is a 
nationally eminent scholar or educator, met several times each year during 
the initial stages to provide technical review and guidance to the effort. 

A local advisory committee composed of Quincy businessmen and civic leaders 
met monthly during the early stages to consider requirements of the physical 
plant and plan for graduate placements. 
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The official title of the program is "Development and Evaluation of an 
Experimental Curriculum for the New Quincy (Mass.) Vocational -Technical 
School." However, the code name "Project ABLE" is being used as descriptive 
of the basic purpose of the study. That purpose is to provide every student 
not in the college preparatory program with an opportunity to achieve compe- 
tence in each of three areas: 

1. Skills and knowledges in a chosen field of work; 

2. The individual's role as a citizen; 

3. Independent pursuit of self-fulfillment and new learning. 

The curriculum for the new school is intended to extend from the 10th 
to the 14th grade, to include post-12th grade instruction in areas like 
electronics, computer data processing, and the machine tools technology. 

The specific job families around which both the new school building and the 
curriculum are being organized are the following: 

1. Business Education — Secretarial, Clerical, Bookkeeping, Sales; 

2. Computer Data Processing — Equipment Operations, Programming; 

3. Electro-Electronics — Electrical Installation, Electronics Repair; 

4. Foods Preparation — Food Service, Food Processing; 

5. General Piping — Plumbing, Pipefitting, Refrigeration; 

6. General Woodworking — Carpentry, Patternmaking, Boatbuilding; 

7. Graphic Arts — Printing, Commerical Art, Drafting; 

8. Health Occupations — Medical, Dental, and Personal Care; 

9. Home Economics — Homemaking, Home Services; 

10. Metals and Machines — Sheet Metal, Machine Shop, Foundry; 

11. Power Mechanics— Auto Body, Auto Mechanics. 



Approach : The principal goal of the project is to demonstrate increased 

effectiveness of instruction that derives content on explicit analysis of 
desired behavior after graduation. Rather than taking a total body of 
knowledge and drawing content from it, curriculum wi 1 1 be defined by what 
technology and industry need for job success. Subordinate objectives em- 
bodied in this plan are the following: 

1. Development of educational objectives . The intent here is to identify 
the behaviors which are desired of the student when he has completed a partic- 
ular course of instruction. Education has no meaning in the abstract; objec- 
tives need to be stated in specific operational terms. While emphasizing 
the vocational area of educational goals, the goals include the development 
of individual attitudes toward work, habits of work, and standards of excel- 
lence. 



2. Derivation of curriculum requirements . Curriculum needs are de- 
scribed in terms of topics within each "subjcect" and placed in an instruc- 
tional sequence which takes prerequisite knowledges systematically into 
account. Each learning sequence is in the curriculum because it must be 
there if the student is to be competent and the justification for its pres- 
ence can be demonstrated on the basis of relevance to a vocationally oriented 
educational goal. Project ABLE, by analyzing the requirements of many jobs 




2 



ittii 









m 
































within each of the eleven broad vocational areas for common and related 
skills, will provide education in the skills and knowledges which are common 
to a variety of occupations. This should minimize the amount of "new" 
training that might result from job change or as opportunities open up 
in related areas and provide flexibility needed to accommodate to changes 
in demands of the technology. 




3. Description of needs for prerequisite learning . The elaboration 
of a new curriculum for the vocational -technical school will also make pos- 
sible the specification of prerequisite knowledges to be acquired in junior 
high years of schooling, including the kinds of student preparation which 
might be gained in industrial arts and other basic areas of instruction. The 
aim is the development of broad exploratory programs in the junior high grades 
to prepare students for productive educational and vocational careers. 

4. Effecting Changes in Student Viewpoints . The new school, with 

its newly designed educational offerings, should become attractive to students 
of a variety of backgrounds and abilities. A most difficult task facing any 
student and his family is that of choosing realistic life goals and the edu- 
cational path to that goal. The pressures of our society have been directed 
toward college attendance while the trade school course has been relegated 
to second-class status. But we cannot afford to allow even a small fraction 
of the 75% who will not go to college for one reason or another to leave 
high school from a "general" course with no particular skill or knowledge to 
market. The project includes preparation of anorganized program for assess- 
ing the student's abilities and interests, and for helping him and his family 
evaluate these over a wide range of occupations. This involves the in-service 
training of junior high school guidance counselors and the provision of 
materials and information for junior high students. 

5. Individualizing Instruction . It has been demonstrated repeatedly 
that individuals differ with respect to a great variety of abilities. The 
traditional classroom has not made sufficient provision for these individual 
differences but with increasing frequency, especially at the elementary level, 
schools are changing to ungraded study programs. Project ABLE incorporates 
the concepts of individualized instruction by providing a framework which 
will allow for maximum flexibility of student progression through a course. 
Learning is a process guided by the teacher rather than forcing facts into 
students. A student's achievement is the standard of his learning progress 
and at the same time a primary source of his motivation. The student is given 
a set of objectives which tell him all the things he is expected to be able 
to do after completing an assignment. Individual tutoring, small group 
discussions among students at a similar stage of progress, demonstrations, 
and possible seminars are decided upon by the teacher and scheduled as needed. 
The key feature, however, is that students do the learning largely on their 
own and student-teacher interactions are not restricted to lecture classes. 
When the student completes an assignment and feels he is ready to go on, he 
informs the teacher who administers the appropriate achievement test, scores 
the test immediately, and if the student has succeeded, makes another assign- 
ment. 
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6, Student Evaluation . Appropriately derived topic objectives 
lead directly to measures of student performance. It is desired here that 
all "units" of instruction have performance measures which are available to 
the student, to instructors, and to guidance counselors. These proficiency 
tests are an essential and integral part of individualized instruction, and 
they contribute to making the student evaluation file a clear history of 
learning achievement. Emphasis in this testing is on attainment of goals, 
rather than upon differentiation of students into "good" or "bad," and to 
provide directions for future effort on the part of the student. 

7. Program Evaluation . A comprehensive program of evaluation 
includes objective measures of immediate outcomes, as well as the foundation 
of techniques for the later collection of follow-up data on educational 
outcomes after graduation. Student evaluations yield many of the basic 
data for program evaluation; this requires systematic recording and storing 
of indicators of student experience and performance. A second feature is 
establishment of techniques for following up the student at periodic intervals 
to collect information on employment, job success, and career progression 

at intervals after graduation. Systematic information of this sort will 
constitute the basis for program evaluation in terms of its long-range effects. 





Figure 1. Overview of Steps in Curriculum Development and Evaluation 
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Some Expected Outcomes! Although the immediate goal has been curriculum 
development, there are features being incorporated that deserve separate 
identification. The students of immediate concern in the project are those 
who are not cc 1 lege -bound. Provision has been made, however, for those who 
may change their educational path. The curriculum provides the courses 
needed to meet, in part, the college entrance requirements; at the same time 
the curriculum is oriented toward the general vocational education that all 
students should have to meet the demands of everyday living in an increasingly 
complex technical society. At the junior high school level is a vocational 
guidance program which helps the student make educational and vocational 
decisions and provides information about the world of work and its occupa- 
tions. . This provides each student with materials he needs to make a sequence 
of decisions narrowing from a choice of educational paths most compatible with 
his abilities and interests through selection of a vocational area to a 
specific group of jobs within the area. At the high school level, a sequence 
in Social Studies deals with the economics and sociology of industry and the 
citizen in modern society in relation to family and political institutions. 

For Mathematics, the courses are aimed at competence and speed in basic 
operations, progressing to measurement, graphing, and descriptive geometry, 
and culminating in algebraic equations and trigonometry. English has as 
its goal to provide instruction in reading, oral and written communication, 
beginning at basic levels of competence and progressing to the comprehension 
of technical reports, and providing appreciation of human values in litera- 
ture, beginning at points of interest appropriate to the least "literate" 
student. A Science course provides the theoretical background for the voca- 
tional areas. Elective courses will be oriented toward acquaintance with 
activities of a self-ful If i l lment leisure-time nature. 



A second benefit which might be expected to accrue follows from the 
idea of general vocational competence. A vocational area-behavioral analysis 
approach to specification of course and topic objectives stated in terms of 
performances required for job execution distinguishes the learnable skills 
and knowledges general izable across several jobs within an occupational group. 
Given training in those skills and knowledges common to a variety of occupa- 
tions, the student-graduate should have a greater flexibility in shifting with 
changes in industry. There is opportunity for increasing specialization, but 
it is built upon a broad base of more general competence within each voca- 
tional area. 



A third outcome of the development effort will be a planned set of graded 
levels of specific education within each area, requiring a range of preparation 
times designated by jobs (or job clusters). The domain of jobs in an occupa- 
tional group has been structured to reflect the progression of skills inherent 
in those jobs. Selection of jobs to represent the area reflects the levels 
involved so that there are clear points of demarcation where a student can 
attain certified competence up to the level commensurate with his individual 
abilities. This concept of multiple exits at points equivalent to the 10th, 
11th, 12th, 13th, and 14th years of education and a specific usable skill 
for each student when he leaves full-time school is an integral part of 
Project ABLE. 
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Summary : Some readers have probably already recognized certain elements 

of the project as familiar ideas reported on elsewhere. For example, the 
job- task approach to the behavioral -performance analysis was used effectively 
in certain military training research programs. Forms of individualized 
instruction and the ungraded school have been used with some frequency at 
the elementary level, and also in secondary schools in Melbourne, Florida; 
Norridge, Illinois; Richmond, California and others. Specifying objectives 
is a long recommended procedure in curriculum development. New York City 
school officials recently made a point on the high school graduates with 
general diplomas as persons "all dressed up with no place to go." The Willis 
Harrington report on education in Massachusetts included comments related to 
a range of content offerings at various ability levels, preparing persons 
to expect major job changes, and for curriculum materials which recognize 
individual differences. Project ABLE differs from other present vocational 
education development efforts in that it represents a multi-faceted approach 
by incorporating all of these concepts, principles, and recommendations in a 
single program to begin when the new school opens rather than making an op- 
erational application of one or two at a time. An ambitious undertaking, 
yes, but one which may well become a model for vocat ional -techni cal education 
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ABSTRACT 

A* Submitted jointly by: Quincy Public Schools (Robert E. Pruitt, Super- 

intendent; American Institutes for Research (John C. Flanagan, President) 

v < 

B. Principal Investigator : Robert M. Gagne, Director of Research, 

American Institutes for Research 

C. Title : DEVELOPMENT AND EVALUATION OF AN EXPERIMENTAL CURRICULUM 

FOR THE NEW QUINCY (MASS.) VOCATIONAL-TECHNICAL SCHOOL 

D. Objectives : The principal goal of the project is to demonstrate increased 

effectiveness of instruction whose content is explicitly derived from anal- 
ysis of desired behavior after graduation, and which in addition attempts 
to apply newly developed educational technology to the design, conduct, and 
evaluation of vocational education. Included in this new technology are 
methods of defining educational objectives, deriving topical content for 
courses, preparation of students in prerequisite knowledges and attitudes, 
individualizing instruction, measuring student achievement, and establishing 
a system for evaluating prog?:am results in terms of outcomes following grad- 
uation. 

E. Procedure : The procedure begins with the collection of vocational information 

for representative jobs in eleven different vocational areas. Analysis will 
then be made of the performances required for job execution, resulting in 
descriptions of essential classes of performance which need to be learned. 

On the basis of this information, a panel of educational and vocational schol- 
ars will develop recommended objectives for a vocational curriculum which in- 
corporates the goals of (1) vocational competence; (2) responsible citizen- 
ship; and (3) individual self-fulfillment. A curriculum will then be designed 
in topic form to provide for comprehensiveness and also flexibility of cover- 
age, for each of the vocational areas. Guidance programs and prerequisite 
instruction to prepare junior high students will also be designed. Selection 
of instructional materials, methods, and aids, and design of materials, when 
required, will also be undertaken. An .important step will be the development 
of performance measures tied to the objectives of ins mction. Methods of in- 
struction will be devised to make possible individualized student progression 
and selection of alternative programs, and teacher-training materials will be 
developed to accomplish inservice teacher education of Quincy School personnel. 
A plan will be developed for conducting program evaluation not only in terms 
of end-o f-year examinations, but also in terms of continuing follow-up of out- 
comes after graduation. 

F. Time Schedule : Beginning 1 January 1965 

Ending 31 December 1969 

G. Budget : Total cost #917*960 

Total Federal Funds requested #623*550 
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1. PROBLEM 



In April of 1963$ the Division of Vocational Education of the Massa- 
chusetts Department of Education issued a report (1963) » hich recommended 
to the Quincy School Committee and Quincy administrators that a new voca- 
tional-technical school be established* Since that time, rapid progress 
has been made* The School Committee has approved preliminary plans and 
an architect has been engaged* Original concepts have been further de- 
fined (1964) to establish a basic division of the curriculum around fam- 
ilies of related jobs* There is a particular concern with the education 
of the culturally disadvantaged who may manifest limited capabilities for 
learning* A particularly challenging part of the total problem relates to 
the overcoming of initial cultural and educational disadvantages which in 
their cumulative effects may show themselves in lowered student motivation 
and achievement* 

Organizational planning is currently concerned with: 

1* Achieving an appropriate balance of conceptual to manual 
'skills in technician training* 

.2* Providing adequate opportunity for students to learn re- 
lated and relatable skills and subject matter without 
' sacrificing vocational learning* 

3* Individualized scheduling* 

4* Providing a differentiated curriculum on a continuum from 
practical to theoretical* 

5* Programming with sufficient flexibility to permit vertical 
and horizontal transfers* 

6* Making cooperative arrangements with business and industry 
that will provide valuable learning experiences which cannot 
be provided in the school* 
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Three factors conspire to make the Quincy vocational-technical school 
of great potential importance to vocational-technical education research! 

1, The full brunt of modern technological change is upon American 
business and industry, but no parallel dynamic has yet bhrst 
upon public vocational education, 

2, Major attention is currently upon vocational and technical educa- 
tion from a variety of disciplines and major institutions in our 
society. The breath of change is in the wind. The opportunity 
for innovation and breakthrough has never been better, 

3, Educational technology has developed rapidly in the last few years 
and now represents a real potential resource if it can be appro- 
priately tapped for the peculiar needs of vocational and technical 
education. 

It is within this context that Quincy school administration has taken 
an experimental approach to vocational— technical education with the approval 
of state and local levels. It is within this context that the building of a 
new school will provide a unique opportunity for innovation. The curriculum 
for the new school is intended to extend from the 9th to the l4th grade, to 
include post-12th grade instruction in such vocational areas as electronics, 
metals, and computers. 

The program and architecture of the new vocational-technical school in 
Quincy will be built around the concepts of job families. The school building 
is being designed by Caudill, Rowlett, and Scott, an outstanding firm of school 
architects who have demonstrated leadership in the development of schools with 
flexibility of function. Outstanding features of the design of the building 
are provisions for flexible placement of internal walls and for adequate above- 
ceiling spacing of utilities cabling. In general, it may be said that the pos- 
sibilities of innovation and experimentation have been carefully incorporated 
into building design. It is here proposed that support be provided for the 
next logical step — development and evaluation of a new curriculum based on 
an explicit derivation from the criterion performances desired of graduates. 







2. DESCRIPTION 



The American In ;titutes for Research and the Quincy Public Schools 

• . - * » • .* 

propose to col3.aboratc in the execution of an experimental program In- 
volving the development and evaluation of a newly designed vocational 
curriculum’, based upon modern educational technology, and to continue 
this cooperation during a five-year period. 

The proposal which follows emphasizes the use of techniques for 
(l) developing vocational and technical course objectives based upon 
a behavioral analysis of the requirements of present and future jobs 
in the job families selected by the Quincy Public Schools; (2) selec- 
ting and designing instructional materials for topic objectives within 
these areas; (3) developing measures of performance to assess student 
progress; and (4) carrying out an evaluation of the program. In brief, 
the intent is to bring to bear some of the most promising new ideas in 
educational technology to the design, development, and tryout of a mod- 
ern curriculum for the education of high school students who are oriented 
toward careers in which college bachelor's degrees are not involved. 

A. Developing Objectives 

The importance to the success of an educational venture of clearly 
defined objectives has been emphasized by various investigators of edu- 
cational problems, including Tyler (1949, 1964), Dressel (1954), Bloom 
and his collaborators (1956), among others. At the level, of college 
instruction, for example, teachers who worked at the process of speci- 
fying objectives are reported to have achieved great clarification in 
the content and method of course presentation, resulting in improved 
student achievement. When such objectives are made clear to the student, 
his interest and motivation are likely to increase. In addition, of 
course, well-defined objectives provide a basis for realistic appraisals 
of student achievement and progress. Examples of the relevance of objec- 
tive definition to several varieties and levels of education are provided 
by Mager (1962). A striking exami>le occurs in recently developed materials 
for elementary science developed by the Commission on Science Education, 

A.A.A.S. (1964). 










», In that field of specialized vocational ^education usually called mil- 
itary training, there have been many examples of the success of techniques 

for deriving educational objectives by "analyses based upon careful descrip- 

’ ' * • . . 
tions of job requirements* Research personnel of the American Institutes 

* •' 4 • 

for Research have been involved in* such studies over a period of years, as 
indicated in the summary by Folley, Fairman, and Jones (I960)* Another 
relevant account of the use of procedures to define military training re- 
quirements is given by Crawford (1962)* 

B* Selecting and Designing Instructional Materials 

The modern trend in developing curricula and instructional materials 
seems definitely to be away, from their determination by an individual author 
or teacher* The past decade has witnessed the completion of a remarkable 
number of newly designed courses in science, mathematics, social studies, 
and other fields, carried out by teams or committees of scholars (cf. Heath, 
1964). It is somewhat rarely, however, that this method of curriculum de- 
velopment begins with clearly defined objectives (for an outstanding excep- 
tion, see Commission on Science Education, 1964)* An alternative method is 
to use .the expert judgments of experienced educators and scholars in defin- 
ing not only the ultimate objectives of some total entity like a "course," 
but als6 the subobjectives (in terms of student achievement) which contrib- 
ute in a sequential fashion to the attainment of such objectives* The 
feasibility of such definition has been shown in the field of mathematics 
by Gagne and his collaborators (1962), by Kersh (1965), Hively (1963), and 
in other fields by Schutz, Baker, and Gerlach (1964), among others. Once 
the structure of a course or topic has been specified in this manner, it 
then should be possible to select instructional materials which are relevant 
to each subobjective, from a variety of sources, or to design them should 
they be found missing. 

C* Developing Measures of Student P rogress 

Effective instruction in vocational education, as in other educational 
areas, is a function of the adequ£*te measurement of educational achievement. 
Achievement measures need to be "scored" in behavioral terms so that they 
can be used to evaluate the degree to which objectives have been attained 
(Bloom, et al, 1956). The important conception that achievement scores need 
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to be referenced to criteria (that is, to objectives) rather than to norms 
..occurs in the writings of Flanagan (1951), Ebel (i960), and Glaser and Klaus 
(1962), among others. 



Data from Project TALENT (Flanagan, et al, 1964) provide baselines for 
both vocational and college-bound students in high school under currently 
used curricula and instructional procedures. Such data can serve as refer- 
ences for the evaluation of new varieties of instruction. Project TALENT 
tests also exemplify the attempt to build direct meaning about student per- 
formance into many of its scores, such as the comprehension of reading cer- 
tain types of materials, the speed and accuracy of solving mathematics prob- 
lems, the rate of acquiring specific items of new knowledge, which provide 
models for the development of appropriate instruments for measuring student 
progress. 

D. Evaluation 



Procedures for evaluation of educational outcomes on both immediate and 
followup bases have been provided by Project TALENT (Flanagan, et al, 1962). 
Modifications and refinements of these procedures are currently being employed 
in an A.I.R. study of the Process and Product of Vocational Education in the 
United States. In both these studies, students are followed up one, five, and 
ten years after graduation from high school to collect information concerning 
their employment, jobs, job satisfaction, salaries, participation in community 
affairs, hobbies, and other factors. It is then possible to relate these var- 
iables to those pertaining to student abilities and school characteristics. 
Based upon these techniques, the present proposal includes the design of a 
system of continuing followup of graduates of vocational education programs 

'j 

to determine educational outcomes following graduation. 



E. Individual Differences 

One of the most striking findings of Project TALENT has been the demonstra- 
tion of the enormous range of individual differences within any single grade of 
a large comprehensive high school. In one such higfh school, for example, it 
was found that on a test of general information, the lowest 10% of the seniors 
were exceeded in score by 73% of the 9th graders. Similar findings obtain to 
those students who attend vocational high schools, insofar as the range of in- 










dividual differences is concerned* Various methods are currently being 
used to try to overcome the difficulties in classroom-based instruction 
produced by such large differences in individual abilities* Perhaps the 
best known of these is "homogeneous grouping," which has sometimes been 
judged to be moderately successful, sometimes not* Obviously, though, the 
most direct type of solution would be one that instructed each individual 
in accordance with his own abilities and previous knowledge, at the same 
time taking into account his interests and career plans* 

It is believed that these innovations, taken together, will lead to 
optimal effectiveness of the vocational education program if they are in- 
tegrated by means of procedures which effect individualized instruction . 
The aim of this development is to give students individual responsibility 
for their own learning, oriented toward realistic and understandable ob- 
jectives which can be incorporated as personal goals* The results of this 
kind of innovation are expected to be as spectacular as the somewhat sim- 
ilar efforts have been reported to be in other high schools* Of particu- 
lar interest is the report of Rollins (1962) on the Middletown Project* 
Brown (<1963), reporting his experience with the "nongraded high school" 
in Melbourne, Florida, makes the following statements (pp. 61-63)1 

"In the nongraded school the intellectual pace of various students 
is more separate and unequal than in graded education* As the cur- 
riculum is expanded and becomes variegated, achievement becomes a 
hallmark of the school* 

"When we began to ungrade the high school at Melbourne, one of the 
earliest observations of the effects of change was a difference in 
the attitude of the student toward learning* Almost overnight, stu- 
dents began to take the initiative for their education away from the 
teachers* Not only did their attitude toward learning improve, but 
their behavior at school underwent an amazing transformation* 

"An analysis of the dropout situation at Melbourne lends additional 

credence to the value of gradeless education* Each year of non- 
grading has seen the number of dropouts decrease* While dropouts 
on the national level are continuing at 30 P er cent, the figures 
at Melbourne last year had gone down to a mere 4 per cent*" 
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3. OBJECTIVES 

Project Goals 

The principal goal of the project is to demonstrate increased effec- 
tiveness of instruction whoso content is based upon explicit derivation 
from analysis of desired behavior after graduation. The subordinate goals 
which are embodied in the plan are as follows: 

1* Development of educational objectives . The intent here is to 
identify the behaviors which are desired of the student when he has com- 
pleted a particular course of instruction. These objectives will be stated 
in specific, operational terms. While emphasizing the vocational area of 
educational goals, they will include the development of individual attitudes 
toward work, habits of work, and standards of excellence. They will also 

-Vil 

give due consideration to the goals of self-fulfillment and good citizenship. 

2. Derivation of curriculum requirements . Curriculum needs will de- 
rived by an explicit and rigorous method, described in terms of topics within 
each '’subject,” and placed in an instructional sequence which takes prereq- 
uisite knowledges systematically into account. Particular attention will be 
paid to the overcoming of individual deficiencies which may represent the cu- 

t 

mulative effects of cultural or educational deprivation. 

3« Description of needs for prerequisite learning in junior high years . 
The elaboration of a new curriculum for the vocational-technical school will 
also make possible the specification of prerequisite knowledges to be acquired 
in junior high years of schooling, including the kinds of student preparation 
which might be gained in industrial arts and other basic areas of instruction. 
The aim of this description of preparatory instruction will be to make pos- 
sible the development of broad exploratory programs in the junior high grades 
by the Quincy schools, to prepare students for productive educational and vo- 
cational careers. 

4. Effecting changes in student viewpoints toward the new school . The 
new school, with its newly designed educational offerings, should become at- 
tractive to students of a variety of backgrounds and abilities. To insure 

y 

that 9th grade students will make suitable choices, a special information and 
guidance program directed to this end will be undertaken. This involves the 
inservice education of junior high school guidance counselors, and the provi- 
sion of materials and information for junior high students. 





















5« Individualizing instruction * A set of procedures will be devised 
which encourage the student to take responsibility for his own learning, and 
to pursue specific instructional objectives which he understands and accepts. 
This outcome in turn loads to the tailoring of instruction, within limits, to 
meet individual student needs. 

6. Student evaluation . Appropriately derived topic objectives will 
lead directly to measures of student performance. It is desired here that 
all ’’units" of instruction have performance measures which are available to 
the student, to instructors, and to guidance counselors. The student eval- 
uation file should be a clear history of learning achievement. 

7« Program evaluation . Student evaluations will yield many of the basic 
data for program evaluation. A comprehensive program of evaluation will in- 
clude other objective measures of immediate outcomes, as well as the founda- 
tions of techniques for the later collection of follow-up data on educational 
outcomes after graduation. 

Vocational Education and Its Goals 

The goals of the proposed project itself are conceived as part of a larg- 
« 

er framework for education at the secondary level, pertaining particularly to 
those students who will not obtain a college bachelor's degree, and who com- 
prise a large proportion (75-80%) of the school population. As the model of 
Figure 1 shows, such education is not conceived as being narrowly vocational, 
but rather as designed to produce effective and well-adjusted citizens for the 
modern world. Accordingly,, the design of curricula and instructional proce- 
dures is intended to place suitable emphasis upon the need for generalizable 
knowledge having the aim3 of responsible citizenship, self-fulfillment as an 
individual, as well as flexibility of vocational choice* in the face of chang- 
ing occupational patterns. 

As the figure indicates, the individual student approaches the choice of 
education at the secondary level influenced by factors in the community and 
his family, and by guidance obtained within the school system. An initial 
appraisal is made of his skills and knowledge for the purpose of guiding his 
tentative choice of a course of study. The school needs to be concerned not 
only with the individual's vocational goals but also those pertaining to his 
functioning as a citizen and as an individual concerned with fulfillment as a 
person. The instructional program of the school is devoted to establishing 












Figure !• A model for modern vocational education 




in the individual the skills, knowledges, and values that he needs to be- 
come an effective individual in society* It makes provision for those whose 
vocational education turns into a professional one at the college level, 
those who will achieve the competence of technicians, and those who will be- 
come skilled operatives in industrial or service occupations* In addition 
it makes provision for the achievement of educational objectives for the in- 
dividual student, irrespective of cultural or educational limitations which 
may affect his level of intellectual functioning* Ultimately, the process 
is expected to prepare the individual as an effective.,- adult, leading a satis- 
fying and successful life in all three areas of vocation, responsible citizen- 
ship, and individual self-fulfillment* 

Specific Outcomes 

The findings and outcomes of the proposed program are expected to pro- 
vide a demonstration of national significance regarding the improvement of 
the status and conduct of vocational education* More specifically, the ex- 
pected outcomes may be outlined as follows: 

1* The demonstration of applicability of newly developed educational 
technology to an important enterprise in vocational education* Included in 
this tecTmology are methods of defining educational objectives, deriving course 
content, individualizing instruction, measuring student achievement, and eval- 

r 

uating program results* 

2* Demonstration of the feasibility of highly flexible planning of vo- 
cational education for the individual student, incorporating goals of voca- 
tional competence, including positive attitudes toward work, effective work 
habits, and standards of performance* In addition, the goals of responsible 
citizenship and individual self-fulfillment will be incorporated and illus- 
trated by a model providing a concrete description of such a system* 

3* Increased amounts of student motivation and achievement, related in 
unusual ways to background and ability factors* 

4* Development and application of techniques of providing instruction 
which takes full account of individual differences in ability, interest, and 
prior learning* 

5* Development and application of new materials for student guidance 
in the junior high years, in preparing students to take advantage of the 
opportunities offered by sound vocational education compatible with their 
interests and abilities* 
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6» Demonstration of high amounts of vocational competence and versa- 
tility on the part of graduates of vocational and technical courses. 

T» Design and establishment of a continuing system for evaluation of 
vocational education in the Quincy Public Schools in terms of procedures 
for assessing outcomes following graduation. 

8. A set of reports and associated instructional materials which 
will account for the study and its findings, intended for the widespread 
dissemination of practical techniques, results, and conclusions. 



4. ADMINISTRATION 

It is proposed that the American Institutes for Research become the 
executive agent for the program, receiving and disbursing the funds required 
for its accomplishment. 

f 

Plans for. the organization of the program are illustrated in Figure 2. 

As the figure indicates, four main elements in the organization of the pro- 
gram are planned: (l) a staff of research investigators experienced in edu- 
cational Research in the areas of vocational and technical education, pro- 
vided by A.I.R. ; (2) experienced educators of the Quincy Public Schools, some 
of whom will devote full time as members of the research team to plan and se- 
lect instructional materials and methods, while others will serve as consult- 
ants in vocational education in particular fields; (3) a panel of university 
scholars who will meet periodically to review statements of educational objec- 
tives, and. to give general guidance in the means of achieving project goals; 
and (4) supervisors of the Massachusetts Division of Vocational Education, 
and also personnel of the Division of Vocational and Technical Education, 

U. S. Office of Education, who will function as advisors in the planning of 
vocational education in various fields. 
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Figure 2* Organization of the Program 
and relation to other agencies* 
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1* A.I.R . Staff . The American Institutes for Research will provide a 
staff of research people who have had extensive experience in the fields of 
educational research, development of educational tests and measures, voca- 
tional education, technical training, learning and retention. An account 
of some of the organization^ experience in these fields is described under 
a Section 8, Facilities. 

A project director will be appointed together with two associates and 
other assistants as required. It is expected that these personnel will be 
experienced in the research fields of education and educational psychology. 
One or more members of the staff will be located in Quincy to maintain con- 
tinuous coordination with the Superintendent and various members of his staff 

For the development of instructional materials, an experienced staff 
of educators will be needed to represent all of the vocational and subject- 
matter areas. It appears that half-time service with the project as em- 
ployees of the project staff, can most readily be combined with half-time 
employment as teachers or supervisors in the Quincy school system. Since 
there are eleven vocational areas, as well as the basic subjects of mathe- 
matics, science, English, social studies, and the arts, a total of sixteen 
half-time people will ultimately be needed. These individuals will serve 
as coordinators of the efforts of curriculum development and instructional 

r 

procedures design in their respective areas, and will also prepare materials 
for inservice teacher education. As their work for the project phases out, 
it is anticipated that they would become full-time members of the vocational- 
technical school staff. 

2. Quincy Public Schools . The Superintendent of the Quincy Public 
Schools, and the Assistant Superintendent for Vocational-Technical Education 
will guide the project from a policy point of view (together with an A.I.R. 
representative), advised by the project* s panel of advisors and consultants, 
and by the professional assistance of the A.I.R. staff. As stated above, 
coordinators of instructional development will be employed half-time as 
members of the school staff. As project employees, they will carry out fre- 
quent consultations with selected teachers in the various areas of vocational 
instruction for review of objectives, selection of relevant materials, and 
planning of instructional procedures. 
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5. Panel of educational scholars . A panel of seven scholars will he 
established to meet at intervals throughout each year of the project, for 
the purpose of reviewing instructional objectives and providing more general 
guidance to the project. While it is difficult to specify exactly the 
qualifications these people should have, it is believed they should possess 
a strong interest in vocational education, an understanding of the needs 
of the students who may not seek or attain a bachelor's degree, and repre- 
sentative knowledge of education in English, mathematics, social science, 
and science as basic or related subjects in the vocational curriculum. 

k. Consultants in the State agency and Federal Office . Considerable 
assistance has been provided to the Quincy schools in the formulation of 
plans for a new school by supervisors of the staff of the Massachusetts 
Division of Vocational Education, as well as by its Director. It is con- 
templated that continued assistance will be sought from this Division in 
consultation on the design of curricula for the various areas of vocational 
education. 

It ip expected that periodic reviews of progress of the program will 
be made by the Division of Vocational and Technical Education, U. S. 

Office of 'Education. Open lines of communication will be maintained to 
representatives of this Office, and it is anticipated that the program 
will benefit by suggestions from this source. The competencies of the 
staff of this Office will be utilized to the greatest extent possible within 
the framework of the program's objectives. 
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5 . PROCEDURES 




a. General Design 

The procedures to be followed are shown in outline form in Figure 3. In 
brief, they are as follows. Analysis will be made of the vocations chosen 
for emphasis in the Quincy program, to determine what kinds of knowledges 
and skills are now required and likely to be required in the future for 
these job families. With the aid of panels of consultants including voca- 
tional and other educational scholars, sets of objectives will be derived 
and stated in operational terms for each vocational area, to include those 
general objectives pertaining to self-fulfillment and responsible citizen- 
ship. Following this, curriculum topics will be delineated in proper se- 
quence, to permit optimal flexibility of student programs. Instructional 
materials will be selected to meet each topic objective, and some may need 
to be newly developed. At the same time, performance measures will be de- 
veloped in conformity with course and topic objectives. Finally, a plan 
will be devised to establish procedures for evaluation of the program, in- 
cluding the collection of follow-up data on graduates. 

(l) Behavior analysis . This step will begin with the collection of 
job information for representative jobs within the job families chosen for 
emphasis in the Quincy program, namely: 

a. Electro -electronics 

b. Metals and machines " V " v "- 

c. Power mechanics 

d. General wood working 

e. General piping 

f. Foods preparation 

g. Computer data processing 

h. Health occupations 

i. Graphic and commercial arts 

j. Home economics 

k. Business education 

To some -xtent, detailed information on jobs within these areas is already 
available as a result of efforts on related projects; however, some new in- 
formation will need to be collected. It may be emphasized that the details 
of job performance are not usually available in systematic published form, 
but must be collected and organized. 
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Figure 3. Overview of Steps in Curriculum Development and Evaluation 
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Beginning with Job descriptions, analysis will be made of performances 
required for job execution, to distinguish the learnable skills and knowl - 
edges from (a) previously acquired basic skills (like hand printing) on the 
one hand, and from (b) highly specialized and specific job skills (like 
using a specialized instrument) on the other. Once identified, these 
learnable entities will be stated in objective language; for example, "Making 
rough sketches to show locations of electric outlets within floor layouts.” 
This analysis will include relevant varieties of performances involving 
personal interaction, suoh as "Reassures patient concerning discomfort of 
procedures to be followed in hospital tests" (for a nurse). The results 
of this analysis will be a set of statements representing the performances 
required in jobs within each job family. It is expected that work habits 
and standards of performance will be included as findings of the analysis. 

(2) Development of course objectives . Prom the statements of required 
performances will be derived a set of objectives for each course of study. 
Explicit rationales will be used to relate the learnable skills and knowl- 
edges to more general statements which represent the goals of instruction 
within each department. At this point, the objectives will be discussed and 

reviewed by a panel of vocational educators and educational scholars, in 

« 

order to determine their appropriateness and comprehensiveness. The result 
of this effort should be a set of clearly understandable goals which can be 
used for universal communications of course objectives, to educators and 
public officials, to parents, and to students. 

Additional consideration will be given by a panel of educational scholars 
to some of the broader goals of vocational education which need to be re- 
flected in a core program of education. These include objectives which may 
be considered to reflect (a) responsible citizenship, and (b) individual 
self-fulfillment. These aspects of individual development should not be 
neglected within the orientation toward narrow vocational goals. Again, 
however, the attempt will be made to derive objectives which are identifiable 
and communicable to all concerned. 

The areas of the curriculum which are considered to contribute broadly 
useful capabilities are those of (l) mathematics, (2) science, ( 3 ) English, 
w social studies, and ( 3 ) the arts. Specific objectives need to be de- 
termined for these subjects which will reflect the goal of establishing 
broadly applicable and generalizable competencies for the individual student, 
regardless of his occupational choice. 








The description of objectives needs also to make adequate allowances 
for individual differences among students, and for flexibility of student 
progress within the educational system. Within given time limitations, 
for example, students who have elected the electro-electronic course of 
study should be permitted to progress as far as they can, and not be held 
back by rigid objectives applicable to particular grade levels. This im- 
plies that an ordered series of objectives may be needed, implying recog- 
nizably different degrees of progression for different students. 




(3) Delineation of topic objectives . Once course objectives have 
been carefully defined, statements of topics and topical sequences within 
each course of study can be specified. In a sense, each of these state- 
ments may be considered a subordinate objective, consisting of a statement 
such as "Converts common proportions into decimal numbers." There are two 
primary reasons for the description of topic objectives. First, such a 
procedure is designed to insure comprehensiveness of coverage, and pre- 



cision of coverage, of needed learnable skills and knowledges within the 
educational program. Second, it is designed to provide for flexibility of 
student programs of study, to allow for individual student needs and abilities. 



Topic objectives will be reviewed by experienced teachers and voca- 
tional educators to insure that they adequately reflect the goals of instruc- 
tion within each field. 



(4) Guidance and information for junior-high students . Another aspect 
of preparation for the conduct of instruction in the new Quincy school is 
important for the success of the program, and needs to be initiated con- 
currently with curriculum planning. This is guidance and instruction for 
potential high school students, currently in junior -high, with the aim of 
establishing suitable appreciation of the educational opportunities to be 
offered by the new school. This effort should be a continuing one during 




; 




the course of the experimental project, but needs to begin with specially 
prepared information and materials designed to bring about positive attitudes 
toward the new educational program. The following activities should be in- 
cluded : 

(a) Providing inservice orientation and instruction for guidance 
counselors and teachers at the junior-high level, particularly 
grade 9 . 
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fb) School and community information dissemination to emphasize 

novel approaches to vocational education and the opportunities 
provided by them. 

(c) Preparation of materials for use of junior-high guidance 
counselors, providing information for students on the new 
program and its relation to employment opportunities. 

(5) Selection of instructional materials , methods , and aids . Topic 
objectives provide a kind of detailed set of "course outlines" for which 
materials of instruction can be selected. The selection will require the 
active participation of teachers who have extensive knowledge of available 
materials in each field. The aim of such selection is to obtain the best 
possible instructional materials, including texts, workbooks, and audio- 
visual presentations, for the achievement of objectives in each topic of the 
course. The criteria used for selection should include the following: 

(a) Each "unit" of instruction will make possible self-study on 
the part of the student. in attaining specified objectives. 

(b) Each topic’s materials should fit in sequence with others 
• that precede and follow them. 

(c) It should be possible for the student to identify the ob- 
jective of each topic and to recognize when he has mastered it. 

The kinds of materials selected for each topic will probably vary con- 
siderably. Textbook or workbook chapters will undoubtedly be frequently 
chosen, and may require some revision to insure their integration into 
suitable sequences. Consideration will be given to available self- 
instructional programs, when these are appropriate to objectives. In addi- 
tion, film strips and motion pictures will be selected to fill particular 
needs of course structure. 

Shop exercises, of course, will also form a part of the materials 
being selected. In particular, the aim in such selection should be to in- 
sure an optimal arrangement for the practice of skills in the shop, based 
upon consideration of previously acquired skills and knowledges, either 
"basic" or "related. " 
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(6) Development of required materials . In all probability, it will 
be found that detailed topic objectives cannot always be met by. the use of 
existing materials in their exact form. Revision of these materials will 
be necessary to accomplish three main purposes: 

(a) Provide transitional text relating each topic to foregoing 
prerequisite topics, and to more general objectives of the 
course. 

(b) Design for self-study, so that each step in learning is 
contained within the materials themselves. 

(c) Provide a means by which the student can determine his own 
attainment of the objective. 

It is believed that materials for some of the topics will need to be 
newly designed, since it is not at all certain that existing texts or 
workbooks contain all of the material which will be found necessary on the 
basis of the analyses previously described. Such materials, too, will be 
designed to meet the criteria stated above. 

Thorough consideration will be given during this stage of the project 
to the projected effectiveness of various audio-visual devices as means of 
accomplishing the desired instruction. Two general classes of device ap- 
pear to deserve special attention, namely, inexpensive cartridge -loading 
filjn projectors designed for individual use, and simple cartridge loading 
tape recorders. Both of these appear to have promising potentialities for 
technical instruction involving media which present materials that are not 
printed language. More detailed specification of media requirements must, 
however, await the planning of instructional objectives. On the basis of 
such progress, it is likely that support will be sought for an additional 
project having the purpose of a systematic evaluation of relevant audio- 
visual media. 
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( 7 ) Development of performance measures . On the basis of topic objec- 
tives, measures of performance will be designed to be administered to stu- 
dents as each topic is completed. Such measures of achievement will conform 
with acceptable test design procedures, but will also incorporate the fol- 
lowing characteristics: 
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(a) Measures will emphasize performance, including the performance 
of basic skills, related skills, and shop skills. Verbal recog- 
nition or recitation of verbal sequences will be eliminated 
except when they are specifically a part of topic objectives. 

(b) Test items will primarily be designed to assess whether the 
individual can or cannot accomplish the performances specified 
by topic objectives. In doing so, they will make the outcomes 
of assessment perfectly clear to both teacher and student. 

(c) Other test items, carefully distinguished from those of (b) 
will be designed to assess the "breadth" of student knowledge 
(that is, its generalizability for the individual student) in 
relation to specific topic objectives. The purpose of such 
measurement, in contrast to (b), is to indicate the potential 
of the student in progressing to advanced parts of the course 
of study. (Some students need to "branch out," whereas others 
need to stay on a relatively narrow track. ) 

(d) Emphasis in testing is on the achievement of recognizable 

# goals, rather than upon the differentiation of students into 
good" or "bad." In a sense, the individual performance 
pleasure is designed to be viewed by the student as something 
he has to demonstrate he can do. It is not designed to demon- 
strate that one student is "smarter" than another. (Of course, 

. the measures described under (c) may actually reflect such 
differences. ) 

(8) Development of instructional procedures . The utilization of in- 
structional materials reflecting realistic topic objectives, together with 
their associated performance measures, can best be put to use within an 
educational framework which allows for great flexibility of student pro- 
gression throughout a course of study . What this implies is a high degree 
of individualization of instruction, permitting each student, within estab- 
lished limits, to pursue the course of instruction which best fits his 
capabilities and interests. Individualized instruction has been accepted as 
a desirable goal of American education for a great many years. There is every 
reason why one should try at this juncture to come as close to achieving it as 
is humanly possible. 
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The important characteristics of a plan for individualizing instruction 
appear to be these: 

(a) The student must be put in the position, and continually in 
the frame of mind, of accepting his own individual responsi- 
bility for learning. Knowledge is something he himself must 
acquire, not that which is "shoved down his throat." 

(b) Defining objectives of instruction (both course and topic) in 
terms which are utterly plain to the student should go a long 
way toward making these the student’s own goals. 

(c) Similarly, it is made clear to the student that performance 
measures are designed to permit him to demonstrate his acquired 
competencies. This procedure is ejected to contribute not a 
little to his motivation to learn. 

(d) Measurement of performance is carried out at frequent intervals 
for the specific purpose of permitting teacher, student, and 
guidance counselor to know what progress has been made, as well 
as what directions of future effort should be planned for the 

. individual student. 

(e) Guidance takes on a new and added dimension within such a 
framework. It becomes a matter of relating student progress to 
his individual goals, as well as to more general course objec- 

v.. lives, at each step of the way. It may be that this kind of 
"guidance" should be to a large extent a teacher function. In- 
forming the student about his progress, about next things to be 
undertaken, relating each curricular choice to course goals 
and to vocational goals constitutes an important and essential 
part of what is meant by individualizing instruction. 

(f) Planning individual alternatives within courses of instruction 
for the individual student becomes a major task for the teacher. 
Most teachers would like to do things this way; all that is 
needed to make it possible is determination and a suitable set 
of convenient instructional materials and tests. Student A, 
let us say, has taken a test on a particular topic and has not 
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been able to demonstrate competence* He must be advised to 
repeat the topic, perhaps using alternate materials* Student 
B has passed the test, indicating that he is ready to progress 
to the next topic* Student C has likewise passed the test, 
but his performance also indicates that he is able to general- 
ize his knowledge widely* He may be expected to accomplish 
the next topic objective rapidly, and may then be led to choose 
additional materials which will enlarge the scope of his under- 
standing and proficiency within his chosen area* 

(9) Instruction for Quincy school personnel * The development of 
curricular and instructional plans will have still another outcome* This 
will be the preparation of materials to be used for the inservice educa- 
tion of Quincy teachers, guidance counselors, and other school personnel. 
The coordinators of development efforts in the various areas of the cur- 
riculum will have attained considerable experience and competence in the 
experimental program* They can develop the necessary materials, and also 
serve as leaders for summer workshops or seminars designed to prepare 
teachers of the new school for their functions and procedures in instruc- 
tion* The conduct of such summer activities for teachers in the Quincy 
School System would in any case be planned as a normal part of preparation 
for the new school* 

(10) Planning and conducting program evaluation . Immediate evalua- 
tion of the program will be provided by performance measures and records 
of student progress. It will doubtless be possible to show remarkable 
differences in individual rates of progress as a result of individualizing 
procedures. Furthermore, it may be expected that individual students of 
high innate abilities will be able to accomplish objectives extending be- 
yond the limits of the school program itself. As for those whose abili- 
ties are more limited, it may be expected that steady progress toward the 
attainment of goals can be demonstrated, perhaps at somewhat slower rates. 

Comprehensive examinations need to be planned and designed for "topic 
sequences," covering a larger range of knowledges and skills than are con- 
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tained in any single topic. Such examinations may roughly serve the pur- 
pose of “end of course” examinations, but they would not necessarily be 
administered on particular dates. In any case, the use of such measures 
will be designed to assess retention of job-related knowledges and job 
skills. 

An additional aspect of evaluation will employ measures of school- 
associated and out-of-school activities as indicators of the effects of 
the program on attitudes, values, and interests of students. The appli- 
cability of such measures as participation in job— related activities, 
outside reading, types of recreation, and others, will be explored. 

A plan will be developed for evaluation of the new educational pro- 
gram in terms of outcomes following graduation. One feature of such a 
plan should be the systematic recording and storing of indicators of stu- 
dent experience and performance during his years of high school and tech- 
nical school attendance (if applicable). A second and most important 
feature is the establishment of techniques for following up the student 
at periodic intervals, so that his location is known. This will make 
possible* the obtaining of information on employment, job success, career 
progression, and so on, at later intervals after graduation. Systematic 

0 

information of this sort will constitute the basis for program evaluation 
in terms of its long term effects. 
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FOREWORD 



This report, submitted in compliance with Article 3 of the contract, 
reports on technical activities of Project ABLE during its eighth quarter 
of operation, 1 April through 30 June 1966. A brief overview of the 
project is presented first, followed by a report summary. The major por- 
tion of the report is a discussion of the development of performance 
measures to be used to assess students' achievement of the objectives of 
instruction. 
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OVERVIEW: Project ABLE 



A Joint Research Project of : Public Schools of Quincy, Massachusetts 

and American Institutes for Research 



Title : DEVELOPMENT AND EVALUATION OF AN EXPERIMENTAL CURRICULUM FOR 

THE NEW QUINCY (MASS.) VOCATIONAL-TECHNICAL SCHOOL 



Object I ves : The principal goal of the project is to demonstrate increased 

effectiveness of instruction whose content is explicitly derived from 
analysis of desired behavior after graduation, and which, in addition, 
attempts to apply newly developed educational technology to the design, 
conduct, and evaluation of vocational education. Included in this new 
technology are methods of defining educational objectives, deriving 
topical content for courses, preparation of students in prerequisite 
knowledges and attitudes, individualizing instruction, measuring student 
achievement, and establishing a system for evaluating program results 
in terms of outcomes following graduation. 

Procedure : The procedure begins with the collection of vocational infor- 

mation for representative jobs in eleven different vocational areas. 

Analysis will then be made of the performances required for job execution, 
resulting in descriptions of essential classes of performance which need 
to be learned. On the basis of this information, a panel of educational 
and vocational scholars will develop recommended objectives for a vocational 
curriculum which incorporates the goals of (1) vocational competence; 

(2) responsible citizenship; and (3) individual self-fulfillment. A 
curriculum then will be designed in topic form to provide for comprehen- 
siveness, and also for flexibility of coverage, for each of the vocational 
areas. Guidance programs and prerequisite instruction to prepare junior 
high students also will be designed. Selection of instructional materials, 
methods, and aids, and design of materials, when required, will also be 
undertaken. An important step will be the development of performance 
measures tied to the objectives of instruction. Methods of instruction 
will be devised to make possible individualized student progression and 
selection of alternative programs, and teacher-training materials will 
be developed to accomplish inservice teacher education of Quincy School 
Personnel. A plan will be developed for conducting program evaluation 
not only in terms of end-of-year examinations, but also in terms of con- 
tinuing follow-up of outcomes after graduation. 

Time Schedule : Begin 1 April 1965 

Complete 31 March 1970 

Present Contract to 30 September 1969 



REPORT SUMMARY 
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This report describes the problems encountered while designing, devel- 
oping, and Implementing an experimental curriculum in a vocational -technical 
school. There are several dangers inherent in such a discussion because there 
is always a tendency to blame the Inevitable inadequacies of the experimental 
program on the other fellow. 

* 

The failure to meet specific requirements of project development derives 
from a number of sources, some of which can be attributed to gaps or loop- 
holes in the proposed methodology and some of which are the result of the 
unexpected. When each step is satisfied according to the plan, the resulting 
product usually has a greater chance of being implemented effectively in a 
live school setting. However, any design that fails to leave room for ad- 
justing to the unforseen is lacking In itself. The realities of the impil- 
mentatlon phase almost always reveal significant gaps in the design and de- 
velopment phases— the largest gap usually being the failure to prepare an 
adequate design for implementation. 

The initial enthusiasm associated with involvement in an experimental cur- 
riculum must be maintained through the tedious work of development and implemen- 
tation. When any part of the process Is separated from the whole, the loss of 
perspective which develops results in actions which often negate or impede the 
forward movement of the effort. The "old" ways are always near, and may be 
retrieved to fill any gap appearing in the new design. Although this may be 
necessary at times, such fixes tend to be retained rather than serving as a 
temporary filler until the innovative step is formalized and tested. Experi- 
ence has demonstrated that this often becomes the case and it further diverts 
the outcomes from the original project goals. The description of the problem 
should be followed by suggestions for solving the problem. However, In most 
instances, the solution can be derived from the original specifications of the 
project methodology. Thus, there is a tendency to rewrite previous discussions 
which deal with specific aspects of the problem. In fact, the problems usually represent 
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deviations from the proposed scheme. When methodology is properly applied, the 
resulting product will probably meet the rigid criteria established within the 
framework of original project specification. 

The problems discussed in the report are divided into four major sections: 

I: General Curriculum Development 

II: Development of Specific Learning Units 

III: Implementation 

IV: Tryout and Revision 

In some cases, problems associated with one section reappear, or may be 
solved in later sections. There are recurrent problem trends which persist, 
however, regardless of attempts to control or eliminate them. Many of these 
are associated with personnel change occur I ng through the development phase. 
Particular sets of skills and attitudes are required to maintain consistent 
progress toward project goals. Training new personnel or re-training persons 
who have been on leave from the project for any duration limits the effective- 
ness of the products. 

It is necessary to point out from the beginning that the responsibility 
for failure in any given dimension is a joint one. Inexact or incomplete co- 
ordination yields results which lack structural coherence and strength. Once 
the problems have been defined clearly, preventive courses of action can be 
specified and implemented. 

It is hoped that the resulting report will specify problem dimensions with 
enough clarity to yield constructive resolutions for those attempting similar 
project efforts. 
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GENERAL CURRICULUM DEVELOPMENT 



The procedures for curriculum development have been outlined and described 
in the Quarterly Technical Report series. Each has dealt with a particular 
step in this process; for example, the problem of defining objectives, the 
roles, characteristics and development procedures for measures of individual 
achievement, the development of learning units, the sequencing of learning units. 
In each report, alternative ways to complete each step are considered and one 
single approach is recommended. 

A number of problems may be incurred as one moves from the theoretical 
level to the operational reality. An attempt is made to identify those factors 
at the source of each problem discussed. 



Translating Theory into Products for Classroom Use 

The logical derivation of learning unit materials follows a rather precise 
set of steps. Each step is a delineation of a preceding step, and in some cases, 
the successful development of a step is almost entirely dependent upon the 
successful completion of the preceding step or series of steps in the process. 

Individuals may be more or less qualified to complete a given step; 
competence in one phase of development is no guarantee of success in completing 
subsequent phases. Application of the curriculum development procedures pre- 
sumes an understanding of the theories and principles on which they are based. 

One persistent problem is that it is difficult if not impossible to provide a 
given model which accounts for all types of conditions and settings. Each 
vocational area and each academic area is defined by its own requirements and 
conditions. Behavioral objectives for one area may require very different 
cognitive processes and student abilities than another area. The kinds of 
behaviors which may be specified will depend to a large extent on the nature 
of the content and student capabilities selected as critical in the job hier- 
archy. 

Some steps are much more difficult to realize than others. Initially, 
the definition of tasks is an objective procedure. In addition to the experience 














in the trade and In teaching, and the expertise provided by the advisory board, 
there are a number of significant sources available to use as reference mate- 
rials such as the Dictionary of Occupational Titles and the Occupational Out- 
look Handbook . 

Objective tasks are more easily defined, particularly when the constraints 
imposed by the school (for example, type of training and equipment, size of 
facility, number of students) are specified. This applies to the statement of 
tasks, the job descriptions and job outlook. However, any number of course 
objectives may be derived from the required tasks. Alternative course objec- 
tives may be acceptable to fulfill the task requirements. Thus, a selection 
from among several viable alternatives becomes the prerogative of the coordin- 
ator writing the objectives. There is often the feeling that to state an ob- 
jective in precise behavioral terms leads to restricted student activity or 
sets a behavior which may become outdated or obsolete in the very near future. 
Either of these feelings neglects to recall the basic principles for the pro- 
ject development, that is providing for flexibility and change by establishing 
a consistently derived and comprehensive framework. 

Once the course objectives have been established, the derivation of topic 
objectives poses similar problems. In teasing out prerequisite behaviors for 
course objectives, many behaviors become more difficult to define; that is, 
they deal with actions frequently defined in education as "understanding," 
"appreciating," "having a grasp of." All of these can be translated into 
something which is observable to some degree. This is one of the most difficult 
parts of translating theory Into a product. Arguments about the "intangibles" 
suggest that non-measurable things are in fact desirable outcomes of any par- 
ticular course of study. Although this may be true in the sense that our powers 
of observing and measuring certain aspects of human behavior are limited, we 
must specify and deal with those within our power to assess as outcomes and 
assume that if these required behaviors are accomplished, the so-called in- 
tangibles will evolve as an out-growth of them 

This problem occurs with varying intensity in the vocational or academic 
subject areas. In general, all areas experience difficulty in accomplishing 
the step, particularly when an attempt is made to specify something other than 
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an overt observable behavior. There Is difficulty expressing in written 
form what a person knows or has experienced. 

The degree to which objectives are clearly stated, effects the quality 
of the student evaluations designed to measure the student's proficiency in 
achieving stated objectives. When the behavioral objective is not clear, the 
items which measure the behavior are not clear and may not measure the desired 
behavior adequately. 

Learning unit objectives are derived from topic objectives. Since these 
will be the outcome toward which ail learning unit experiences are directed, 
they too must be stated in concise, behavioral terms which* will preview or 
suggest the learning activities. The problems associated with stating these 
objectives and writing specific learning units are described in a later sec- 
tion. 

Although the relationship between tasks, course objectives, and topic 
objectives is defined, an individual working on one step may isolate that part 
of the effort from the overall process. This tends to lessen the impact of 
a product which should evolve from preceding steps and reflect their logical 
continuity. 



Required Coordination Between Area Specialist and Educational Research Personnel 

* 

The development of learning materials relies to a large extent on the 
comprehensive analysis just described. The extent and kind of coordination 
required by the effort is often difficult to predict prior to development. 

Some of the factors which influence this include: experience and extent of 

information possessed by vocational or academic area specialist, the amount 
of time which can be devoted to the task, the confidence of the person in the 
process which has been outlined and the willingness to cooperate with members 
of the project who advise on methodology and approach. If these characteristics 
can Identified and evaluated prior to selecting individuals, a considerable 
amount of real progress can be made before the process begins. Working with 
subject matter experts may best be described as a case of individualized in- 
struction or learning. Each person requires different amounts and kinds of 
technical assistance, motivation and reinforcement to progress. 






Adequacy of Reference Materials Which Describe the Curriculum Procedures 



Numerous publications exist which relate to procedures for curriculum 
development and evaluation. With few exceptions, they are directed to a 
rather limited audience, that is those who have been directly involved in the 
process or those who intend to pursue it on a fulltime basis. The faculty 
member who has become a member of the curriculum team often needs what might 
be described as a cookbook. This document describes the ingredients and the 
steps involved from beginning to end. These must be prepared as excerpts from 
project reports or as special documents which give examples, explanations and 
summaries as needed. It is necessary to test the adequacy of these directions 
and supplementary information provided. This is accomplished through direct 
feedback from the individuals involved in developments after they have demon- 
strated that the materials provide sufficient direction and information. 

Identifying Necessary Qualifications of Personnel Involved in Project Development 

Many different types of skills are required to complete all steps in the 
curriculum development. Comprehending the overall objectives and plans of the 
project, sharing information with other department members, conducting the 
research, writing objectives and creating materials— each requires distinct yet 
related skills. Persons who do well in one phase will not necessarily perform 
all other steps in a satisfactory manner. 

In some cases, individuals are requested to develop materials for a voca- 
tional area range which is beyond their direct experience. It is not always 
feasible to release another person within a department who has the information * 

necessary to fill the gap. This means that in some instances a coordinator 

i 

must obtain the additional data in any way that he can, rather than in a syste- 
matic manner. Thus, the chain is weakened, and further development builds on 
a less than firm foundation. 

There is a tendency for individual teachers to view the development effort 
In terms of their teaching- training experience. The most persistent carry-over 
seems to be the tendency to write for the teacher rather than for the student.^ 
Traditional lesson plans, teacher guides and course objectives have the teacher 
as the primary audience even though they are attempting to satisfy the individ- 
ual needs of students. When the teacher attempts to describe what the student 
does, he usually ends up telling what the teacher does. 

4 










Writing In clear and concise terms is a problem. Some individuals have 
considerable difficulty in transcribing their experience into words which 
can be comprehended by students. There is a tendency to assume that a student 
will "know" certain things and thus, prerequisite behaviors are not always 
specified. Also, individuals who have learned and performed things in one 
way tend to repeat the procedure when they write the learning experiences. A 
given procedure may be appropriate, but it must be evaluated in terms of the 
behavioral objective, not in terms of an established precedent. Coordinators 
must be flexible in thinking of classroom situations. In order to have a 
student develop a stated capability, a teacher may have to do a number of dif- 
ferent things rather than any single action such as a lecture or a demonstra- 
tion. As a director of learning, the teacher must use many approaches to assist 
the student in the learning process. 



Wiiiinqness to Participate in Change 

Individuals within a given school system have a background of experiences 
and have brought a considerable degree of effort to bear on the problems of the 
teaching- learning situation. In the view of the instructor, what remains is 
the "reliable 11 result of personal research carried on through the years. Cer- 
tain methods, materials, and approaches have been tried and rejected for one 
reason or another. Others have been tried and continued for other reasons. 

Some have been retained and modified to fit a given set of conditions. In 
general, the current operating procedures have been derived as a result of 
experiences In and out of the classroom. 



When asked to completely re-think the problem of learning, there are vary- 
ing degrees of, resistance. Individuals may be quite satisfied with their pro- 
ducts to date, either In terms of graduates or of a good classroom environment. 
Although the criteria may not be rigorously defined, they are sufficient to 
encourage the "status quo" at least for a while longer. Others have no qualms 
about beginning ail over again, using the experiences they have gained. The # 
enthusiasm, however, seems to continue only until the individual is asked to 
generate things which can "fit" into the already established pattern. When 
an Individual must, for example, begin from a completely new starting point 



i 



5 



..r* 



O 

ERIC 









mm 



„ w~ . ■ ... g/UgjjUc 



I 



I 

I 



or basis, or reformulate an approach, such that he Is on "unfamiliar 5 ' grounds, 
there Is a tendency to try and defend what has been, rather than use It to 
produce a different set of teaching* learning experiences. 

Whenever an experiment is attempted, a certain risk is involved. There 
Is always a chance that the new product will be only as good or worse than 
the existing one. The uncertainty associated with the situation Is often 
detrimental to development and encourages a reiteration of "tried methods." 
These facts reinforce the notion that a firm conviction to project goals and 
methods must precede any development. 



Establishin g Priorities in a Complex School System 

Even with unlimited resources and complete flexibility within a school 
system regarding faculty, students, and the scheduling of classes, the problems 
of curriculum development and tryout would exist. In most cases, a school 
must adopt new techniques within a structure that imposes restrictions and 
constraints. The introduction of new approaches often demands considerable 
shifting, rearrangement and reorganization of staff, and students and physical 
facilities, in the initial phase, the school continues to operate on its 
present basis, while attempting to release personnel for curriculum development 
activities and gear up for the preliminary tryout. 

In the preliminary tryout phase, ail individuals and classrooms may not 
be involved. Although the "new" is being introduced gradually over a period 
of years, the problems normally associated with maintaining an efficient, oper- 
ating school system may be compounded. In effect, two systems are being con- 
ducted simultaneously. Even with intensive and comprehensive orientation to 
the new procedures and products, the day-to-day problems which administration, 
faculty, and students must face and solve still remain. 

Thus, the establishment of priorities becomes a critical activity. Addi- 
tional human resources and funds do not always provide the solution. It is 
difficult to foresee all contingencies. Individuals assigned to one activity 
may be called upon to meet an immediate need for which no one else is qualified. 
Another individual cannot always fill the gap created by an individual who has 
9 developed some expertise in developing curriculum materials— at least, not 
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without a great deal of training and practice. A qualified teacher may have 
to cover a vocational shop and demands of this type may easily draw curriculum 
developers away from their task at a moment's notice. There Is a constant 
reordering of priorities each time a special situation develops, thus reducing 
the overall effectiveness of those involved in the experimental program devel- 
opment. 



Communication among ABLE Staff. AIR, and School Personnel 

Recognized as a critical ingredient in the success of a joint effort, 
adequate and comprehensive communication among project participants becomes 
imperative. It is not enough to maintain an "open door" policy which permits 
the free flow of information only when it is identified "as necessary or 
appropriate" by members of the development team. Frequent, direct contacts 
must be initiated and maintained to preserve the desired continuity, keep a 
current record of progress, both in quality and quantity, and provide almost 
immediate reinforcement and feedback. In this way, an up-to-date assessment 
of success and failure can be used to correct deficiences as soon as they 
occur. These assessments must, in addition, be concrete and direct. They 
must reassure participants of correct application of project methods and 
appropriate selection of content and media. 

Some "less than tangible" types of information and communication are also 
required to boost morale and create a willingness to cooperate in the production 
of quality materials. .Chief among these are frequent positive reinforcement 
and acknowledgment of expertise in a given content area. 



Determination of Information Requirements for Project Personnel 

Too much information can often be as detrimental to overall program 
development as not enough information. The "appropriate" level and type of 
information shared is different for different individuals involved in the 
program. Excess data or information can discourage individuals by its massive 
appearance. The task may appear too complex, with no end in sight, accentu- 
ating the risk involved if a mistake is made. Out of context, constructive 
argumentation or criticism can be misinterpreted as an accusation of individ- 
ual incompetence. The whole-part method of learning information has 
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applicability here. The majority of staff members need only have an outline 
of the total program steps, followed by an intensive description of a partic- 
ular step immediately preceding the work. 



Allocating Time to Each Step of the Development 



In order to assess the amount of time required to complete a given step 
of curriculum development, all the preceding factors must be taken into account. 
Assuming little or no experience on the part of curriculum developers, the 
most difficult phase and the one which consumes the greatest amount of time 
with only a few exceptions appears to be the derivation of course objectives. 

Since this is the hinge on which subsequent steps depend, the specification 
of objectives can legitimately demand additional time, providing they are 
clearly written in concise behavioral terms. If this step is completed ad- 
equately, there are fewer struggles as one begins to delineate sub-objectives 
and derive proficiency measures and learning experiences. This is not intended 
to underestimate the difficulty in developing each subsequent step; the pro- 
ducts which flow from "good" course objectives, however, have a much greater 
chance of success than those which are extrapolated from less than specific 
behavioral objectives. The development of proficiency measures to evaluate 
objectives consumes the second greatest amount of time. Writing items and 
developing evaluation procedures requires a sophisticated application of 
measurement theory and demands instruction and assistance in constructing 
those which adequately measure the objectives. Since evaluation is used to 
assess learning objectives and student behavior, sufficient time must be 
allocated to this step, otherwise, it may be difficult to draw any conclusions 
about the success of the student or the system. 

Although it is necessary to permit some flexibility in allocating time 
to each step of project development, it is imperative that certain deadlines 
be established for each unit of work. In order to progress, a point must be 
established beyond which no further revision can be made for a given step. 

A commitment is then made to the f product as it appears for a given step. This 
prevents a deteriorating regression to former stages of development. Revisions 
will be necessary but, in most cases, can wait until the experience of tryout 
demonstrates the need for it. This is one more 'reason for creating good course 
ob j ect i ves • 
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DEVELOPMENT OF SPECIFIC LEARNING UNITS 



Identif ying Prerequisite Behaviors for Tasks. Course Objectives. Topic 
Objectives, and Learning Unit Objectives 

For efficient learning to take place, it is necessary to identify the 
appropriate behaviors and their prerequisites and to arrange them In a 
logical sequence. To accomplish this, tasks, course objectives, and topic 
objectives must be identified, analyzed, and organized in a hierarchy in 
which the certain behaviors are necessarily preceded by others. This con- 
tinuum is applicable to one capability or a set of highly related capabili- 
ties and should not necessarily be followed for unrelated capabil i ties. 7 For 
example, if it is necessary to differentiate between types of screwdrivers 
before using them, then the differentiation" precedes use. It is unimportant, 
however, if it precedes a different, unrelated capability such as using a 
wrench . 

One of the major difficulties in stating the behaviors associated with 
tasks is that the coordinators are unable to identify which tasks are 
appropriate to given jobs and the requirements of these tasks. There is a 
tendency to exaggerate the complexity of tasks or duties in occupations until 
it becomes difficult to differentiate between specific jobs on a given 
hierarchy. 

The learning units are often sequenced to reflect textbook content rather 
than the required skill hierarchy. The result of this sequencing is not a 
new approach to teaching— it is breaking chapters of books into smaller 
elements. 



Statement of Criteria for Behavioral Objectives 

One of the elements of a complete behavioral objective is a statement 
of criteria which students must meet. This statement should be written In 
clear, precise terms so that the reader will have no difficulty in identify- 
ing the intent of the writer. In addition. It Is written so that the be- 
havior can be observed and evaluated objectively. Terms like "understand, 
appreciate, develop and appreciation of, etc." should be avoided. 







There are a number of factors that make the writing of behavioral ob- 
jectives clearly difficult. One is that in the past, behavoriai objectives 
have not been written clearly. Objectives which state "to appreciate the 
relationship between X and Y" appear in numerous publications used by curric- 
ulum developers. Another is that there seems to be a reluctance to write in 
specific terms for the fear of omitting elements or exceptions which may per- 
tain to a given capability. A third seems to be a desire to make the objective 
sound complex, pedantic, or highly literary. The thought appears to be that 
if the objective is simple and easy to understand it will diminish student 
motiviation to meet It. Finally, the criteria are stated in very nebulous 
terms as exemplified by objectives such as "student will demonstrate activity 
to accepted specifications." 

Identifying Individuai Learning Experiences 

The learning experiences should reflect the various capabilities necessary 
to satisfy the learning unit objective. Their number and content may vary 
from unit to unit. In fact, it seems to be entirely appropriate that a given 
capability can, and should be attained through different kinds of learning 
experiences. For example, if certain knowledge is needed by a student to 
accomplish a given objective, this knowledge may be achieved through reading, 
viewing films or filmstrips, mockups, etc. The experiences should be sequenced 
in the same manner as the learning units themselves— that is, in a logical 
hierarchy. In addition, the experiences should be related to the unit objective 
and be meaningful to the student. 

There are many factors which limit the variety of learning experiences. 

One of these is failure to explore the available alternate experiences. Gen- 
erally, the experiences consist of those activities which are already mentioned 
in i*>ome textbook or manual. Another is the failure to transform a principle 
or theory Into an activity. Still another, is the belief that for any learn- 
ing to take place, the students must be assembled in a group and the teacher 
must lecture to explain the facets of a given concept or capability. 



Performance-Type Activities 

To provide the student with first-hand exposure to vocational capabilities, 
emphasis is placed on activity or performance-centered learning experiences. 

\ i 
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These are augmented by experiences which provide the theoretical support and 
background. Performance oriented activities permit the student to utilize 
both cognitive and psychomotor processes in achieving the objectives of the 
learning unit. They also provide the instructor and the student with the 
opportunity to evaluate the attainment of vocational capabilities. 

Non-performance activities such as reading or listening to a lecture, 
have their value, but these alone cannot provide the student with the capa- 
bilities which he will need in the working world. In addition, many students 
are at relatively low-reading and vocabulary levels, a situation which casts 
doubt on how much the student really learns through these methods. There is 
also a question if portions of theories, principles or other similar infor- 
mation are really needed by the student to achieve given capabilities at 
given skill levels or if they are needed at all. For example, is it necessary 
to teach the student to arrive at a square root of a number by the longhand 
method when square root tables are commonly available? 



Determining how much theory and how much practice is to be included 

in a given learning unit is sometimes a difficult process. Very often, how- 

0 

ever, a clear statement of the learning unit objective will provide the 
answer. Frank and accurate requirements are also essential. 

As with other segments of this effort, it appears that there are dif- 
ficulties in deciding how much emphasis should be placed on theory and how 
much on practice or activity. There is a tendency to stress theory through 
reading and lecture. From this, the student mu:t somehow demonstrate the 
capability to the satisfaction of the instructor. Activity is implied but 
not specified* and since the learning units are written for the student, 



there must be some method of translating the theory into action. How this 
is to be done is not clear, but It appears that it will result in the in- 
structor telling the student or a group of students what to do each step of 
the way. If this is the result of developing learning units (and it can easily 
turn out that way), how does this differ from education before? 

Development of Student-Oriented Materials 

Learning units are developed for the student. It is he who must read and 
translate them into activity. The units must, therefore, be addressed to the 
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student In the language he understands. Activities of learning experience 
should be student rather than teacher-oriented; the student has to observe, 
perform, compare, associate, identify, etc., with the teacher assuming the 
role as an advisor or director. 




The learning units, placing a greater responsibility for learning on 
the student, are also changing the role of the teacher. This role, as student 
director or advisor, requires that the teacher take a different approach to 
the learning process. He is involved in it, perhaps even more actively than 
before, but in a different manner. It Is difficult to adapt to this change, 
since students have been taught (lectured) by teachers for many years. This 
difficulty is carried over into the development of learning units. There is 
a tendency to return to the traditional (teacher-oriented) approach, and the 
result is a learning unit giving directions to the teacher rather than to 
the student. In addition, there seems to be a somewhat strong reliance on 
lecture as a substitute for learning experiences. The argument here is that 
there is activity going on (i.e. , lecturing by the teacher). 




Perhaps one way to ensure that the units will be developed for the student 
is to write them in second-persOn-si ngular form. When this approach is used, 
the unit speaks directly to the student, telling what to do and in what se- 
quence it should be done. 

Relationship between Components of Learning Units 




One. of the most critical characteristics of a learning unit is the close 

& 

relationship among Its components. If this relationship does not exist or 
is tenuous, there Is little hope of meeting the objective of any given learn- 
ing unit. 

The starting point is, of course, the learning unit objective. It de- 
termines the content and the parameters of the unit. The various components 
of the learning unit, particularly the learning experiences, should support 
and satisfy the unit objective. Upon completion of the activities prescribed 
by the learning experiences, the student should have in his repertoire, that 
capability stated in the objective. 

\ 

12 

! 































The difficulty in developing units In which the components relate 
closely to each other is determined by the clarity of the learning unit ob- 
jective. The clearer the objective, the easier it is to develop closely 
related components. 

Another problem is the selection of appropriate reference materials and 
learning aids. This requires a consideration of a number of factors, such 
as the relevance of the information, the ability of the student to benefit 
from it, and its availability, cost, and ease of utilization. These must 
constantly be evaluated since the criteria for their selection may change 
from unit to unit. 

The ultimate test of the relevance of learning unit components is the 
student's ability to demonstrate the capability stated in the objective. If 
he can satisfy this criterion, then much of the discussion about the compon- 
ents may be academic. There is another question that must be answered— that 
is, "how efficient is this method?" Is the objective being met at great 
expense to teacher and student time, is it costly, and is there another, more 
efficient method of reaching the goals stated in the objective? This problem 
should be resolved after the learning materials have been tried out and eval- 
uated. 

Relationship Between Learning Units and Student Evaluations 

The student evaluation portion of the learning unit is designed to assess 
whether or not the student has achieved the goal or goals stated in the learn- 
ing unit objective. It consists of a number of paper and pencil items (the 
number varies from unit to unit) and one or more performance measures which 
require the student to demonstrate the capability or capabilities specified 
in the learning unit objectives. Successful completion of these items permits 
the student to move to the next unit in the sequence or hierarchy. Unsuccess- 
ful performance requires a repeat of unit learning experiences or the use of 
alternative or supplementary means to meet the objective. 

The validity of the evaluation depends heavily on the clarity and re- 
latedness of the learning unit objective, learning experiences, references, 
and learning aids. Many of the problems encountered in developing the student 
evaluation portions of the learning units are, in fact, caused by the lack of 
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clarity and relatedness. When items are derived from references which are 
not related to the learning unit objective, then, although they may have face 
validity, it is likely that they are not measuring the behavior. 

Another problem is determining how many items are needed to measure a 
particular behavior. For each learning unit a decision must be made on how 
many and what type items should be written. This decision is guided by the 
nature of the elements which make up the learning unit (objective, experiences, 
references, etc.). One certainty is that performance or activity items should 
be included in every student evaluation. These should require the student to 
act out the behaviors which are stated in the objectives. The performance of 
activities required by the objective should result in a valid assessment of 
student progress. 






! 



Supplementing the performance test are a number of penci l-and- paper items. 
These are developed to assess the knowledges associated with the behaviors 
and to identify students' weaknesses with respect to particular behaviors. 

The assessment of student capabilities should not rely on these items except 
in situations which require cognitive skills and simulate the behaviors ad- 
equately. 

The types of paper and pencil items to be constructed are determined by 
the learning unit. It is difficult or inappropriate, at times, to rely solely 
on multiple choice items. The behaviors may be better simulated by measures 
which require the student to list sequences or match objects with objects, 
activities, rules, or symbols. Drawing symbols, sketches, or diagrams, and 
labeling symbols or objects are examples of paper- and.penci I items which sim- 
ulate behavior very well. 

In summary, there are many problems associated with developing an adequate 
assessment of any given behavior, but there are also many alternative solutions 
available to the item developer. The critical requirement of any evaluation 
is that the behaviors are assessed properly. This can be achieved if the 
behaviors are simulated well and the criteria specified. 
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IMPLEMENTATION 



1 1 1 



Organization of School System to Accommodate Different Learning Situations 

A number of problems arise when a new curriculum does not entirely 
replace the old one. The school then has the problem of administering two 
entirely different approaches or systems of learning. The difficulty is 
further compounded when the two curricula are not compatible in terms of 
scheduling, learning materials, and teacher preparation. 

The learning contexts in a vocational -technical school also present a 
number of problems which probably would not exist or be greatly reduced in 
a strictly academic situation. These consist of a number of items, such as 
equipment, space allocation, work assignments, etc. In addition, the physical 
presence of two groups, unrelated in terms of learning approach, seems to 
hinder the performance of both. 
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When a new curriculum which has little rigid control over student time 
and location is introduced, the students and the instructors appear to have 
some difficulty adjusting it. Different responsibilities for learning are 
placed on the student and on the instructors. Adjusting to this situation 
takes time, training, and a willingness on the part of the school administration, 
faculty, and student body. 

Availability of Materials to Support Learning Experiences 

Shortages of materials, learning aids, and other educational support 
seem to be an inherent problem in all educational systems. In a school system 
which introduces a new curriculum, the problem becomes more acute because the 
new situation requires more and different support (usually more expensive) 
than the old. 



These materials must be identified specifically, and a method should 
be employed to make certain that they will meet their expectations. These 
materials are the means to an end; they support the learning unit only if the 
student requires their use. One of the problems, however, is that when the 
learning units rely heavily on reference materials, learning aids and other 
equipment which are not available to the students, the entire learning effort 
will be seriously hampered. 
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Large-Scale Introduction of Learning Units 

When new learning materials are introduced Into an educational system 
difficulty In implementing them Is almost inevitable. This problem is In- 
creased when individualized materials are adopted by the system. First, the 
amount of materials Is so great In number and bulk that it creates storage 
and distribution problems. Second, since the materials differ substantially 
from those used in previous years, both the instructors and students must be 
trained in their use. (The instructors must also be trained in the organi- 
zation and application of these materials.) Third, is the problem of quantity 
reproduction of materials with limited equipment and small staff. (This prob- 
lem is discussed in a later section.) Fourth, when two systems of learning 
are operating simultaneously in one school as mentioned previously, many dif- 
ficulties arise. Of concern here is the control of materials in any depart- 
ment of the school. This control includes such factors as distribution of all 
learning materials, administration of evaluation measure, recording the re- 
sults of the measures, and keeping an account of student progress. These 
factors require a great deal of paper work on the part of the instructors, and, 
coupled with evaluation forms of several kinds, the task is almost overwhelming 

There is a definite need to plan every detail and anticipate every con- 
tingency when introducing new learning materials into^'an educational system. 
Preparation of instructors in the administration of these materials and keep- 
ing records of student performance and characteristics of materials (appro- 
priateness, student comprehension, validity, etc.) is essential. Many of the 
problems associated with large-scale introduction of new learning materials 
are yet to be identified, and these must be solved after all the pertinent 
data are collected and evaluated. 

Reproduction of Developed and Revised Materials 

Although quantity reproduction of learning materials seems to be a 
straightforward task, there are many problems associated with it. These range 
from the number and kind of equipment and personnel available, to the length 
and complexity of the materials themselves. Problems created by shortages of 
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personnel and materials and Inadequacy of reproduction equipment are obvious. 

The difficulties In administering and coordinating quantity reproduction, 
however, are not as overt. 

First, there must be a delineation of responsibilities In the administra- 
tion of the entire reproduction effort. Such activities as purchasing materials, 
securing and maintaining equipment, hiring personnel, and transporting com- 
pleted materials to their appropriate destinations are clearly administrative 
and clerical problems which should be handl d by individuals trained and 
employed in this area. 

Second, the technical editing and other technical support related to 
the learning materials should be the domain of the technical staff. This staff 
should not be expected to perform the duties of the administrators and clerks, 
as this hampers the efficiency of technical support which Is so critical to 
the content of materials. 

Third, the physical quality of the materials reproduced largely depends 
on the available staff and equipment. At this point, it appears that equip- 
ment is a more critical factor. Staff can be trained rather easily— after a 
few weeks they seem to perform their duties satisfactorily. Equipment, however, 
does not improve with use. While staff members can be trained to perform 
certain specialty tasks associated with quantity reproduction of learning 
materials, a ditto machine cannot be modified to perform as a four-color press. 

Finally, there is the problem of flexibility of response. There are 
times when the demand on the staff for learning materials ts overwhelming, and 
there are times when they are relatively small. A decision must be reached as 
to size of staff which is sufficient to cope with the contingencies which may 
arise at a given time. The content of the learning materials also determines 
the nature of the staff. Since many illustrations appear In the learning units, 
at least one staff member must be able to make the necessary illustrations. 

Those pictures or diagrams which are not within staff capabilities must be sent 
to a source which can reproduce them. 

Staff Preparation for New Conditions of Learning 

Preparation of instructors to operate effectively under new learning 
conditions is of utmost importance. The reasons for this are obvious. The 












actual preparation Itself requires an Intensive effort on the part of the 
Project staff. The Instructors must be provided with Information concerning 
the philosophy of Project ABLE, Its objectives, methodology, etc. The Imp- 
lementation and application of learning materials must be explained In de- 
tail so that there will be no question on the part of instructors as to how 
the materials are to be used In classrooms. 

it is also Important to Involve the Instructors directly in revising 
materials and suggesting the development of entirely new learning units. This 
involvement should facilitate teacher acceptance of Project ABLE materials 
and increase the commitment to implement the learning units. In addition, 
this approach has the potential of increasing the overall efficiency of the 
tryout and generating support for Project ABLE as a whole. 









Yryout ano revision of curriculum materials 



Monitoring the System 

The long and tedious process of curriculum development brings about a 
product which provides specificity, clarity, comprehensiveness, and satis- 
faction of individual student needs. The product has been acquired by using 
a procedure which is at once exciting and exhausting. Although individuals 
may be reasonably proud of their efforts, the process does not stop here. 

The real "test" of the product is Its tryout, evaluation, and revision phases. 

It may be argued that tryout should rely on the expertise of the school 
staff. That is, the teachers are operating in a situation which should be 
more familiar to them. Even though new techniques and approaches are being 
utilized, the director of learning has the support of much of his previous 
teaching experience, particularly regarding course content. In some cases, 
however, the old methods may seriously interfere with the new learning mate- 
rials and methods. 

Preliminary implementation reveals a number of critical problems such 
as those identified in the previous section. The extent to which a system can 
operationally effect the new curriculum should be revealed during this phase. 
Procedures to alleviate the most difficult problems can be identified if the 
system is monitored adequately. 

There is a tendency to feel that once the materials and classroom pro- 
cedures are operational the work, per se,is done. In fact, there is a great 
deal of work involved in monitoring and evaluating the progress or lack of 
progress in ail parts of the program. Many individuals contribute to this 
preliminary evaluation. Each has a responsibility to record the outcomes as 
they are experienced. To assist in this process, a set of evaluation forms 
has been devised and is designed to elicit the responses as soon as action 
occurs. 

One set of forms requests written impressions and progress reports from 
students, teachers, counselors, and administrators. Although this demands 
additional effort, it is a good way to get immediate feedback on what is hap- 
pening. There is general acquiescence regarding the method of gaining such 
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Information, but there is a reluctance to do it. In addition to being time 
consuming, an individual may feel that stating a problem reflects an in- 
adequacy in his efforts, and this reduces the impact of reporting hard-core 

problems. In wondering if their responses will affect their "grades," students 

.. • *?■ 

may cover some of the mo3t significant problems. Therefore, the impressions 
may be less than exact or recorded long after a unit has been completed. In 
either case, there can be a proliferation of the most serious problems. 

the interview technique used for collecting data on program tryout also 
tends to elicit the desired response rather than real descriptions of what 
is occurring. If individual teachers and students answer "fine," the work 
of revision is reduced. There is no substitute for an honest appraisal of 
the program to anticipate and solve the basic problems, but obtaining it is 
not an easy task. 

One way to insure more objective and complete recording is to instruct 
ail staff on who should do what and when. Evaluation forms should then be 
collected individually from staff members to prevent delay or partial com- 
pletion of evaluation. 



Agreement on Content Validity 

When the tryout evaluation has been completed and the responses summarized, 
one possible conclusion may be that several different approaches or types of 
content are suitable. Although this may be true, it is not always feasible to 
implement all approaches. A decision is required, and some agreement must be 
reached. This implies a departmental unity and staff cooperation. The eval- 
uation of the "best" method or learning hardware must always be in terms of 
the original objectives. In some cases, these objectives must be modified 
or changed to satisfy critical needs of students. 

Responsibility for revision must be designated and clearly specified to 

all participating staff members. Teachers and students] wi 1 1 , in the final 

1 

analysis, determine the effectiveness of any new prOgra^n or approach. It is 
they who must make it work . 
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This report has touched upon the most obvious problems which face a 
group of . persons attempting to develop and Implement a new curriculum. It 
has not been possible to Identify all problems or solutions. It should be 
viewed as signals designed to anticipate system difficulties. As such, it 
can serve as a comprehensive outline for a seminar or teacher-training in- 
stitute. Structured properly, the most obvious and common problems could 
be Identified, and steps could be initiated to resolve potential sources of 
difficulty. 

From the delineation of problems, one significant fact emerges— the 
intent and procedures associated with the project must be delineated in con 
cise terms which staff can understand and accept . 



21 



u; n-vxtifj >* tj; rJxraui ■. . vein a.--- -rM.-tt**. -..r* * 





